The diagnostic and triage accuracy of digital and online symptom checker tools: a systematic review (preprint)

ABSTRACT Objective To evaluate the accuracy of digital and online symptom checkers in providing diagnoses and appropriate triage advice. Design Systematic review. Data sources Medline and Web of Science were searched up to 15 February 2021. Eligibility criteria for study selection Prospective and retrospective cohort, vignette, or audit studies that utilised an online or application-based service designed to input symptoms and biodata in order to generate diagnoses, health advice and direct patients to appropriate services were included. Main outcome measures The primary outcomes were (1) the accuracy of symptom checkers for providing the correct diagnosis and (2) the accuracy of subsequent triage advice given. Data extraction and synthesis Data extraction and quality assessment (using the QUADAS-2 tool) were performed by two independent reviewers. Owing to heterogeneity of the studies, meta-analysis was not possible. A narrative synthesis of the included studies and pre-specified outcomes was completed. Results Of the 177 studies retrieved, nine cohort studies and one cross-sectional study met the inclusion criteria. Symptom checkers evaluated a variety of medical conditions including ophthalmological conditions, inflammatory arthritides and HIV. 50% of the studies recruited real patients, while the remainder used simulated cases. The diagnostic accuracy of the primary diagnosis was low (range: 19% to 36%) and varied between individual symptom checkers, despite consistent symptom data input. Triage accuracy (range: 48.8% to 90.1%) was typically higher than diagnostic accuracy. Of note, one study found that 78.6% of emergency ophthalmic cases were under-triaged. Conclusions The diagnostic and triage accuracy of symptom checkers are variable and of low accuracy. Given the increasing push towards population-wide digital health technology adoption, reliance upon symptom checkers in lieu of traditional assessment models, poses the potential for clinical risk. Further primary studies, utilising improved study reporting, core outcome sets and subgroup analyses, are warranted to demonstrate equitable and non-inferior performance of these technologies to that of current best practice. PROSPERO registration number CRD42021271022. SUMMARY BOXES What is already known on this topic Chambers et al. (2019) have previously examined the evidence underpinning digital and online symptom checkers, including the accuracy of the diagnostic and triage information, for urgent health problems and found that diagnostic accuracy was generally low and varied depending on the symptom checker used. Given the increased reliance upon digital health technologies by health systems in light of the ongoing COVID-19 pandemic, in addition to the marked increase in availability of similarly themed digital health products since the last systematic review, a contemporary and comprehensive reassessment of this class of technologies to ascertain their diagnostic and triage accuracy is warranted. What this study adds Our systematic review demonstrates that the diagnostic accuracy of symptom checkers remains low and varies significantly depending on the pathology or symptom checker used. The findings of this systematic review suggests that this class of technologies, in their current state, poses significant risk for patient safety, particularly if utilised in isolation.

Bridging the Digital Divide: A National Survey Assessing Public Readiness for Digital Health Strategies Against COVID-19 within the United Kingdom (preprint)

Digital health technologies are a major feature of contemporary public health strategies, particularly in relation to the COVID-19 pandemic. However, digital initiatives risk excluding vulnerable groups, thereby propagating poor health outcomes. We assessed how groups at higher risk from COVID-19 report their relationships with key digital health initiatives in the United Kingdom. We found that those who are female, over 60 and of a lower social group are less confident in using digital information to make health decisions. Those over 40, from lower social groups and of lower educational attainment use digital resources less often in seeking COVID-19 health information. Lastly, those over 60, from lower social groups and of lower educational attainment are less confident in distinguishing reliable digital COVID-19 information. This suggests that a ‘digital first’ model of COVID-19 pandemic management may exacerbate existing digital and health inequalities by reinforcing barriers to health information and public health services.

DETERMINANTS OF BURNOUT AND OTHER ASPECTS OF PSYCHOLOGICAL WELL-BEING IN HEALTHCARE WORKERS DURING THE COVID-19 PANDEMIC: A MULTINATIONAL CROSS-SECTIONAL STUDY (preprint)

BackgroundThe Covid-19 pandemic has placed unprecedented pressure on healthcare systems and workers around the world. Such pressures may impact on working conditions, psychological wellbeing and perception of safety. In spite of this, no study has assessed the relationship between safety attitudes and psychological outcomes. Moreover, only limited studies have examined the relationship between personal characteristics and psychological outcomes during Covid-19. MethodsFrom 22nd March 2020 to 18th June 2020, healthcare workers from the United Kingdom, Poland, and Singapore were invited to participate using a self-administered questionnaire comprising the Safety Attitudes Questionnaire (SAQ), Oldenburg Burnout Inventory (OLBI) and Hospital Anxiety and Depression Scale (HADS) to evaluate safety culture, burnout and anxiety/depression. Multivariate logistic regression was used to determine predictors of burnout, anxiety and depression. ResultsOf 3,537 healthcare workers who participated in the study, 2,364 (67%) screened positive for burnout, 701 (20%) for anxiety, and 389 (11%) for depression. Significant predictors of burnout included patient-facing roles: doctor (OR 2.10; 95% CI 1.49-2.95), nurse (OR 1.38; 95% CI 1.04-1.84), and other clinical (OR 2.02; 95% CI 1.45-2.82); being redeployed (OR 1.27; 95% CI 1.02-1.58), bottom quartile SAQ score (OR 2.43; 95% CI 1.98-2.99), anxiety (OR 4.87; 95% CI 3.92-6.06) and depression (OR 4.06; 95% CI 3.04-5.42). Factors significantly protective for burnout included being tested for SARS-CoV-2 (OR 0.64; 95% CI 0.51-0.82) and top quartile SAQ score (OR 0.30; 95% CI 0.22-0.40). Significant factors associated with anxiety and depression, included burnout, gender, safety attitudes and job role. ConclusionOur findings demonstrate a significant burden of burnout, anxiety, and depression amongst healthcare workers. A strong association was seen between SARS-CoV-2 testing, safety attitudes, gender, job role, redeployment and psychological state. These findings highlight the importance of targeted support services for at risk groups and proactive SARS-CoV-2 testing of healthcare workers.

What Has Been the Impact of Covid-19 on Safety Culture? (preprint)

Introduction Covid-19 has placed an unprecedented demand on healthcare systems worldwide. A positive safety culture is associated with improved patient safety and in turn patient outcomes. To date, no study has evaluated the impact of Covid-19 on safety culture. Methods The Safety Attitudes Questionnaire (SAQ) was used to investigate safety culture during Covid-19 at a large UK teaching hospital. Findings were compared with baseline data from 2017. Incident reporting from the year preceding the pandemic was also examined. Results Significant increases were seen in SAQ score for doctors and AHPs (p value) from baseline (p value). A decrease in SAQ was found in the nursing group. Largely due to perception of management and safety climate subscales. During Covid-19, on univariate regression analysis, female gender (p<0.001), age 40-49 years (p<0.01), non-white ethnicity (p<0.001), nursing job role (p<0.001) were all associated with lower SAQ scores. Training (p<0.001) and support (p<0.001) for redeployment were associated with higher SAQ scores. On multivariate analysis, non-disclosed gender (-0.13, -0.26-0.00), non-disclosed ethnicity (-0.11, -0.22-0.00), nursing role (-0.15, -0.24-0.06), and support (0.24, 0.07-0.4) persisted to significance. A significant decrease (p<0.003) was seen in error reporting after the onset of the Covid-19 pandemic. Discussion Differences in reported safety culture may reflect perception of risk due to: occupational exposure, job function, or access to support services. Reductions in incident reporting may be due to increased workload, change in nature of work, or changing safety attitudes. Targeted high-quality support for redeployed staff may help improve safety during future pandemics.